1. Field of the Invention
The present invention relates to computer networks and more particularly to load balancing Traffic Engineering (TE) label switched paths (LSPs) of a computer network.
2. Background Information
A computer network is a geographically distributed collection of nodes interconnected by communication links and segments for transporting data between end nodes, such as personal computers and workstations. Many types of networks are available, with the types ranging from local area networks (LANs) to wide area networks (WANs). LANs typically connect the nodes over dedicated private communications links located in the same general physical location, such as a building or campus. WANs, on the other hand, typically connect geographically dispersed nodes over long-distance communications links, such as common carrier telephone lines, optical lightpaths, synchronous optical networks (SONET), or synchronous digital hierarchy (SDH) links. The Internet is an example of a WAN that connects disparate networks throughout the world, providing global communication between nodes on various networks. The nodes typically communicate over the network by exchanging discrete frames or packets of data according to predefined protocols, such as the Transmission Control Protocol/Internet Protocol (TCP/IP). In this context, a protocol consists of a set of rules defining how the nodes interact with each other. Computer networks may be further interconnected by an intermediate network node, such as a router, to extend the effective “size” of each network.
Since management of interconnected computer networks can prove burdensome, smaller groups of computer networks may be maintained as routing domains or autonomous systems. The networks within an autonomous system (AS) are typically coupled together by conventional “intradomain” routers configured to execute intradomain routing protocols, and are generally subject to a common authority. To improve routing scalability, a service provider (e.g., an ISP) may divide an AS into multiple “areas.” It may be desirable, however, to increase the number of nodes capable of exchanging data; in this case, interdomain routers executing interdomain routing protocols are used to interconnect nodes of the various ASes. Moreover, it may be desirable to interconnect various ASes that operate under different administrative domains. As used herein, an AS or an area is generally referred to as a “domain,” and a router that interconnects different domains together is generally referred to as a “border router.”
An example of an interdomain routing protocol is the Border Gateway Protocol version 4 (BGP), which performs routing between domains (ASes) by exchanging routing and reachability information among neighboring interdomain routers of the systems. An adjacency is a relationship formed between selected neighboring (peer) routers for the purpose of exchanging routing information messages and abstracting the network topology. The routing information exchanged by BGP peer routers typically includes destination address prefixes, i.e., the portions of destination addresses used by the routing protocol to render routing (“next hop”) decisions. Examples of such destination addresses include IP version 4 (IPv4) and version 6 (IPv6) addresses. BGP generally operates over a reliable transport protocol, such as TCP, to establish a TCP connection/session. The BGP protocol is well known and generally described in Request for Comments (RFC) 1771, entitled A Border Gateway Protocol 4 (BGP-4), published March 1995.
Examples of an intradomain routing protocol, or an interior gateway protocol (IGP), are the Open Shortest Path First (OSPF) routing protocol and the Intermediate-System-to-Intermediate-System (IS-IS) routing protocol. The OSPF and IS-IS protocols are based on link-state technology and, therefore, are commonly referred to as link-state routing protocols. Link-state protocols define the manner with which routing information and network-topology information are exchanged and processed in a domain. This information is generally directed to an intradomain router's local state (e.g., the router's usable interfaces and reachable neighbors or adjacencies). The OSPF protocol is described in RFC 2328, entitled OSPF Version 2, dated April 1998 and the IS-IS protocol used in the context of IP is described in RFC 1195, entitled Use of OSI IS-IS for routing in TCP/IP and Dual Environments, dated December 1990, both of which are hereby incorporated by reference.
An intermediate network node often stores its routing information in a routing table maintained and managed by a routing information base (RIB). The routing table is a searchable data structure in which network addresses are mapped to their associated routing information. However, those skilled in the art will understand that the routing table need not be organized as a table, and alternatively may be another type of searchable data structure. Although the intermediate network node's routing table may be configured with a predetermined set of routing information, the node also may dynamically acquire (“learn”) network routing information as it sends and receives data packets. When a packet is received at the intermediate network node, the packet's destination address may be used to identify a routing table entry containing routing information associated with the received packet. Among other things, the packet's routing information indicates the packet's next-hop address.
To ensure that its routing table contains up-to-date routing information, the intermediate network node may cooperate with other intermediate nodes to disseminate routing information representative of the current network topology. For example, suppose the intermediate network node detects that one of its neighboring nodes (i.e., adjacent network nodes) becomes unavailable, e.g., due to a link failure or the neighboring node going “off-line,” etc. In this situation, the intermediate network node can update the routing information stored in its routing table to ensure that data packets are not routed to the unavailable network node. Furthermore, the intermediate node also may communicate this change in network topology to the other intermediate network nodes so they, too, can update their local routing tables and bypass the unavailable node. In this manner, each of the intermediate network nodes becomes “aware” of the change in topology.
Typically, routing information is disseminated among the intermediate network nodes in accordance with a predetermined network communication protocol, such as a link-state protocol (e.g., IS-IS, or OSPF). Conventional link-state protocols use link-state advertisements or link-state packets (or “IGP Advertisements”) for exchanging routing information between interconnected intermediate network nodes (IGP nodes). As used herein, an IGP Advertisement generally describes any message used by an IGP routing protocol for communicating routing information among interconnected IGP nodes, i.e., routers and switches. Operationally, a first IGP node may generate an IGP Advertisement and “flood” (i.e., transmit) the packet over each of its network interfaces coupled to other IGP nodes. Thereafter, a second IGP node may receive the flooded IGP Advertisement and update its routing table based on routing information contained in the received IGP Advertisement. Next, the second IGP node may flood the received IGP Advertisement over each of its network interfaces, except for the interface at which the IGP Advertisement was received. This flooding process may be repeated until each interconnected IGP node has received the IGP Advertisement and updated its local routing table.
In practice, each IGP node typically generates and disseminates an IGP Advertisement whose routing information includes a list of the intermediate node's neighboring network nodes and one or more “cost” values associated with each neighbor. As used herein, a cost value associated with a neighboring node is an arbitrary metric used to determine the relative ease/burden of communicating with that node. For instance, the cost value may be measured in terms of the number of hops required to reach the neighboring node, the average time for a packet to reach the neighboring node, the amount of network traffic or available bandwidth over a communication link coupled to the neighboring node, etc.
As noted, IGP Advertisements are usually flooded until each intermediate network IGP node has received an IGP Advertisement from each of the other interconnected intermediate nodes. Then, each of the IGP nodes (e.g., in a link-state protocol) can construct the same “view” of the network topology by aggregating the received lists of neighboring nodes and cost values. To that end, each IGP node may input this received routing information to a “shortest path first” (SPF) calculation that determines the lowest-cost network paths that couple the intermediate node with each of the other network nodes. For example, the Dijkstra algorithm is a conventional technique for performing such an SPF calculation, as described in more detail in Section 12.2.4 of the text book Interconnections Second Edition, by Radia Perlman, published September 1999, which is hereby incorporated by reference as though fully set forth herein. Each IGP node updates the routing information stored in its local routing table based on the results of its SPF calculation. More specifically, the RIB updates the routing table to correlate destination nodes with next-hop interfaces associated with the lowest-cost paths to reach those nodes, as determined by the SPF calculation.
Multi-Protocol Label Switching (MPLS) Traffic Engineering has been developed to meet data networking requirements such as guaranteed available bandwidth or fast restoration. MPLS Traffic Engineering exploits modern label switching techniques to build guaranteed bandwidth end-to-end tunnels through an IP/MPLS network of label switched routers (LSRs). These tunnels are a type of label switched path (LSP) and thus are generally referred to as MPLS Traffic Engineering (TE) LSPs. Examples of MPLS TE can be found in RFC 3209, entitled RSVP-TE: Extensions to RSVP for LSP Tunnels dated December 2001, RFC 3784 entitled Intermediate-System-to-Intermediate-System (IS-IS) Extensions for Traffic Engineering (TE) dated June 2004, and RFC 3630, entitled Traffic Engineering (TE) Extensions to OSPF Version 2 dated September 2003, the contents of all of which are hereby incorporated by reference in their entirety.
Establishment of an MPLS TE-LSP from a head-end LSR to a tail-end LSR involves computation of a path through a network of LSRs. Optimally, the computed path is the “shortest” path, as measured in some metric, that satisfies all relevant LSP Traffic Engineering constraints such as e.g., required bandwidth, “affinities” (administrative constraints to avoid or include certain links), etc. Notably, a TE-LSP with constraints is referred to as a “constrained TE-LSP.” Path computation can either be performed by the head-end LSR or by some other entity operating as a path computation element (PCE) not co-located on the head-end LSR. The head-end LSR (or a PCE) exploits its knowledge of network topology and resources available on each link to perform the path computation according to the LSP Traffic Engineering constraints. Various path computation methodologies are available including CSPF (constrained shortest path first). MPLS TE-LSPs can be configured within a single domain, e.g., area, level, or AS, or may also span multiple domains, e.g., areas, levels, or ASes.
The PCE is an entity having the capability to compute paths between any nodes of which the PCE is aware in an AS or area. PCEs are especially useful in that they are more cognizant of network traffic and path selection within their AS or area, and thus may be used for more optimal path computation. A head-end LSR may further operate as a path computation client (PCC) configured to send a path computation request to the PCE, and receive a response with the computed path, which potentially takes into consideration other path computation requests from other PCCs. It is important to note that when one PCE sends a request to another PCE, it acts as a PCC.
Some applications may incorporate unidirectional data flows configured to transfer time-sensitive traffic from a source (sender) in a computer network to a destination (receiver) in the network in accordance with a certain “quality of service” (QoS). Here, network resources may be reserved for the unidirectional flow to ensure that the QoS associated with the data flow is maintained. The Resource ReSerVation Protocol (RSVP) is a network-control protocol that enables applications to reserve resources in order to obtain special QoS for their data flows. RSVP works in conjunction with routing protocols to, e.g., reserve resources for a data flow in a computer network in order to establish a level of QoS required by the data flow. RSVP is defined in R. Braden, et al., Resource ReSerVation Protocol (RSVP), RFC 2205. In the case of traffic engineering applications, RSVP signaling is used to establish a TE-LSP and to convey various TE-LSP attributes to routers, such as border routers, along the TE-LSP obeying the set of required constraints whose path may have been computed by various means.
In certain network configurations, such as, e.g., symmetric networks, multiple paths may exist that have equal costs between a source node and a destination node (or in MPLS TE, between a head-end node and a tail-end node). An example of a symmetric network is a network including a core system of routers, generally large capacity routers, interconnecting one or more edge devices (edge routers) at equal costs from the core. Known equal-cost multi-path (ECMP) routing techniques may be used to route packets along the multiple paths of equal cost in a symmetric network. One such ECMP technique distributes traffic from the source to the destination substantially evenly across the multiple equal-cost paths so that no single path (e.g., network element or link along the path) is overwhelmed, i.e., to essentially “load balance” the traffic across the paths. An illustrative load balancing technique applies a “round-robin” algorithm that cycles distribution of traffic (e.g., IP packets or new TE-LSPs) from one path to the next in such a way (e.g., ordered or random) that each path receives substantially equal amounts of traffic. For example, if there are two possible paths, a first TE-LSP is established over the first path, a second TE-LSP is established over the second path, a third TE-LSP is established over the first path, etc. While this technique is effective, it is less than efficient at considering TE-LSPs or other traffic already traversing the possible paths, such as from other head-end nodes or routers. Other load balancing techniques that do account for network configurations (e.g., available bandwidth of the paths) are available, as known to those skilled in the art.
Occasionally, a network element (e.g., a node or link) will fail, causing redirection of the traffic that originally traversed the failed network element to other network elements that bypass the failure. Generally, notice of this failure is relayed to the nodes in the same domain through an advertisement of the new network topology, e.g., an IGP Advertisement, and routing tables are updated to avoid the failure accordingly. Typically, both IP traffic and any TE-LSPs are redirected to avoid a failure in a manner known to those skilled in the art.
To obviate delays associated with updating routing tables when attempting to avoid a failed network element (i.e., during convergence), some networks have employed MPLS TE fast reroute (FRR). MPLS FRR is a technique that may be used to quickly reroute traffic around failed network elements in a TE-LSP. MPLS FRR is further described, for example, by P. Pan, et al., in Fast Reroute Extensions to RSVP-TE for LSP Tunnels <draft-ietf-mpls-rsvp-lsp-fastreroute-07. txt>, available from the Internet Engineering Task Force (IETF). According to the technique, one or more links in a primary path are protected links (i.e., they are protected by an alternate path). If a failure occurs on a protected link or node, TE-LSPs (and consequently the traffic that they carry) are locally rerouted onto an appropriate alternate path (e.g., a “backup tunnel”) by the node immediately upstream from the failure. The backup tunnel acts as an FRR path for the primary TE-LSP and obviates delays associated with other measures, such as tearing down the primary TE-LSP after having gracefully rerouted the TE-LSPs affected by the failure, should an alternate path around the failed network element exist. Notably, in some situations (e.g., where MPLS Traffic Engineering is used for the sole purpose of FRR), networks may establish “unconstrained TE-LSPs” for traffic not conventionally contained within constrained TE-LSPs (e.g., IP traffic). An unconstrained TE-LSP is a TE-LSP that has no constraints or affinities (i.e., reserves no bandwidth), and is used generally to supply MPLS TE features to traversing traffic.
Generally, a local FRR may be followed by an optimization in order to force the traffic to follow a more optimal TE-LSP. For example, assume a head-end node has first and second equal-cost paths for TE-LSPs to a tail end node, each with FRR protection. In the event a TE-LSP on the first path has a failure and local FRR is performed (e.g., rerouting the TE-LSP to a backup tunnel, in some cases with a higher cost than the original failed path), the head-end node may later optimize the locally rerouted TE-LSP to traverse the second equal-cost path. Once the failed network element is restored (an “event”), a notification (e.g., an IGP Advertisement) is sent to the surrounding domain so the network can potentially redirect the traffic over the more optimal route (i.e., “re-optimize”). Notably, several known “triggers” (e.g., a timer or an event-driven trigger) can be used to initiate the optimization of a TE-LSP. Moreover, establishing a new TE-LSP and re-optimizing an established TE-LSP are generally referred to herein as “optimizing” a TE-LSP.
One problem with optimizing TE-LSPs, e.g., after FRR, is that for networks with multiple equal-cost paths, the rerouted TE-LSPs generally do not revert to the previous path since that path is no more optimal than the current rerouted path (i.e., they have the same cost). This situation reduces the ability of networks to balance the traffic (load) over the equal-cost paths. Particularly, however, in the case where unconstrained TE-LSPs are utilized, there is currently no method for signifying the number of unconstrained TE-LSPs that traverse a particular path (i.e., links of the path). Even if the unconstrained TE-LSPs had been load balanced previously (e.g., before FRR), there is currently no method for returning the unconstrained TE-LSPs to their load balanced state, resulting in undesirable asymmetrical traffic routing. There remains a need, therefore, for a technique that efficiently load balances TE-LSPs, particularly unconstrained TE-LSPs. There also remains a need for a technique that efficiently load balances TE-LSPs after having been rerouted from a previously load balanced equal-cost path.